Measuring Information from Moments

We investigate the problem of representing information measures in terms moments underlying random variables. First, we derive polynomial approximations conditional expectation operator. then apply these to bound best mean-square error achieved by a estimator—referred here as PMMSE. In Gaussian channels, PMMSE coincides with minimum (MMSE) if and only input is either or constant, i.e., channel given output degree at most 1. By combining I-MMSE relationship, new formulas for (e.g., differential entropy, mutual information) that are As an application, introduce estimators from data via approximating our sample moments. These shown be asymptotically consistent possess desirable properties, e.g., invariance affine transformations when used estimate information.